
            <!DOCTYPE html>
            <html lang="en">
            <head>
                <meta charset="UTF-8">
                <title>Kubernetes 那些奇技淫巧</title>
            </head>
            <body>
            <a href="https://andyoung.blog.csdn.net">原作者博客</a>
            <div id="content_views" class="markdown_views prism-atom-one-light">
                    <svg xmlns="http://www.w3.org/2000/svg" style="display: none;">
                        <path stroke-linecap="round" d="M5,0 0,2.5 5,5z" id="raphael-marker-block" style="-webkit-tap-highlight-color: rgba(0, 0, 0, 0);"></path>
                    </svg>
                    <h3><a id="1__Pod_IP_0"></a>1. 在容器中获取 Pod 的IP</h3> 
<p>通过环境变量来实现，该环境变量直接引用 resource 的状态字段，示例如下：</p> 
<pre><code class="prism language-yaml"><span class="token key atrule">apiVersion</span><span class="token punctuation">:</span> v1
<span class="token key atrule">kind</span><span class="token punctuation">:</span> ReplicationController
<span class="token key atrule">metadata</span><span class="token punctuation">:</span>
  <span class="token key atrule">name</span><span class="token punctuation">:</span> world<span class="token punctuation">-</span>v2
<span class="token key atrule">spec</span><span class="token punctuation">:</span>
  <span class="token key atrule">replicas</span><span class="token punctuation">:</span> <span class="token number">3</span>
  <span class="token key atrule">selector</span><span class="token punctuation">:</span>
    <span class="token key atrule">app</span><span class="token punctuation">:</span> world<span class="token punctuation">-</span>v2
  <span class="token key atrule">template</span><span class="token punctuation">:</span>
    <span class="token key atrule">metadata</span><span class="token punctuation">:</span>
      <span class="token key atrule">labels</span><span class="token punctuation">:</span>
        <span class="token key atrule">app</span><span class="token punctuation">:</span> world<span class="token punctuation">-</span>v2
    <span class="token key atrule">spec</span><span class="token punctuation">:</span>
      <span class="token key atrule">containers</span><span class="token punctuation">:</span>
      <span class="token punctuation">-</span> <span class="token key atrule">name</span><span class="token punctuation">:</span> service
        <span class="token key atrule">image</span><span class="token punctuation">:</span> test
        <span class="token key atrule">env</span><span class="token punctuation">:</span>
        <span class="token punctuation">-</span> <span class="token key atrule">name</span><span class="token punctuation">:</span> POD_IP
          <span class="token key atrule">valueFrom</span><span class="token punctuation">:</span>
            <span class="token key atrule">fieldRef</span><span class="token punctuation">:</span>
              <span class="token key atrule">fieldPath</span><span class="token punctuation">:</span> status.podIP
        <span class="token key atrule">ports</span><span class="token punctuation">:</span>
        <span class="token punctuation">-</span> <span class="token key atrule">name</span><span class="token punctuation">:</span> service
          <span class="token key atrule">containerPort</span><span class="token punctuation">:</span> <span class="token number">777</span>
</code></pre> 
<p>容器中可以直接使用 <code>POD_IP</code> 环境变量获取容器的 IP。</p> 
<h3><a id="2__33"></a>2. 指定容器的启动参数</h3> 
<p>我们可以在 Pod 中为容器使用 command 为容器指定启动参数：</p> 
<pre><code class="prism language-bash">command: <span class="token punctuation">[</span><span class="token string">"/bin/bash"</span>,<span class="token string">"-c"</span>,<span class="token string">"bootstrap.sh"</span><span class="token punctuation">]</span>
</code></pre> 
<p>看似很简单，使用数组的方式定义，所有命令使用跟 Dockerfile 中的 CMD 配置是一样的，但是有一点不同的是，<code>bootsttap.sh</code> 必须具有可执行权限，否则容器启动时会出错。</p> 
<h3><a id="3_Poddocker_43"></a>3. 让Pod调用宿主机的docker能力</h3> 
<p>我们可以想象一下这样的场景，让 Pod 来调用宿主机的 docker 能力，只需要将宿主机的 <code>docker</code> 命令和 <code>docker.sock</code> 文件挂载到 Pod 里面即可，如下：</p> 
<pre><code class="prism language-yaml"><span class="token key atrule">apiVersion</span><span class="token punctuation">:</span> v1
<span class="token key atrule">kind</span><span class="token punctuation">:</span> Pod
<span class="token key atrule">metadata</span><span class="token punctuation">:</span>
 <span class="token key atrule">name</span><span class="token punctuation">:</span> busybox<span class="token punctuation">-</span>cloudbomb
<span class="token key atrule">spec</span><span class="token punctuation">:</span>
 <span class="token key atrule">containers</span><span class="token punctuation">:</span>
 <span class="token punctuation">-</span> <span class="token key atrule">image</span><span class="token punctuation">:</span> busybox
 <span class="token key atrule">command</span><span class="token punctuation">:</span>
 <span class="token punctuation">-</span> /bin/sh
 <span class="token punctuation">-</span> <span class="token string">"-c"</span>
 <span class="token punctuation">-</span> "while true; \
 do \
 docker run <span class="token punctuation">-</span>d <span class="token punctuation">-</span><span class="token punctuation">-</span>name BOOM_$(cat /dev/urandom <span class="token punctuation">|</span> tr <span class="token punctuation">-</span>cd 'a<span class="token punctuation">-</span>f0<span class="token punctuation">-</span>9' <span class="token punctuation">|</span> head <span class="token punctuation">-</span>c 6) nginx ; \
 done"
 <span class="token key atrule">name</span><span class="token punctuation">:</span> cloudbomb
 <span class="token key atrule">volumeMounts</span><span class="token punctuation">:</span>
 <span class="token punctuation">-</span> <span class="token key atrule">mountPath</span><span class="token punctuation">:</span> /var/run/docker.sock
 <span class="token key atrule">name</span><span class="token punctuation">:</span> docker<span class="token punctuation">-</span>socket
 <span class="token punctuation">-</span> <span class="token key atrule">mountPath</span><span class="token punctuation">:</span> /bin/docker
 <span class="token key atrule">name</span><span class="token punctuation">:</span> docker<span class="token punctuation">-</span>binary
 <span class="token key atrule">volumes</span><span class="token punctuation">:</span>
 <span class="token punctuation">-</span> <span class="token key atrule">name</span><span class="token punctuation">:</span> docker<span class="token punctuation">-</span>socket
 <span class="token key atrule">hostPath</span><span class="token punctuation">:</span>
 <span class="token key atrule">path</span><span class="token punctuation">:</span> /var/run/docker.sock
 <span class="token punctuation">-</span> <span class="token key atrule">name</span><span class="token punctuation">:</span> docker<span class="token punctuation">-</span>binary
 <span class="token key atrule">hostPath</span><span class="token punctuation">:</span>
 <span class="token key atrule">path</span><span class="token punctuation">:</span> /bin/docker
</code></pre> 
<p>参考：<a href="https://www.infoq.com/presentations/patterns-microservices-kubernetes" rel="nofollow">Architecture Patterns for Microservices in Kubernetes</a></p> 
<h3><a id="4_Init_container_79"></a>4. 使用Init container初始化应用配置</h3> 
<p>Init container可以在应用程序的容器启动前先按顺序执行一批初始化容器，只有所有Init容器都启动成功后，Pod才算启动成功。看下下面这个例子（来源：<a href="https://stackoverflow.com/questions/44109308/kubernetes-mounting-volume-from-within-init-container" rel="nofollow">kubernetes: mounting volume from within init container - Stack Overflow</a>）：</p> 
<pre><code class="prism language-yaml"><span class="token key atrule">apiVersion</span><span class="token punctuation">:</span> v1
<span class="token key atrule">kind</span><span class="token punctuation">:</span> Pod
<span class="token key atrule">metadata</span><span class="token punctuation">:</span>
  <span class="token key atrule">name</span><span class="token punctuation">:</span> init
  <span class="token key atrule">labels</span><span class="token punctuation">:</span>
    <span class="token key atrule">app</span><span class="token punctuation">:</span> init
  <span class="token key atrule">annotations</span><span class="token punctuation">:</span>
    <span class="token key atrule">pod.beta.kubernetes.io/init-containers</span><span class="token punctuation">:</span> '<span class="token punctuation">[</span>
        <span class="token punctuation">{<!-- --></span>
            <span class="token key atrule">"name"</span><span class="token punctuation">:</span> <span class="token string">"download"</span><span class="token punctuation">,</span>
            <span class="token key atrule">"image"</span><span class="token punctuation">:</span> <span class="token string">"axeclbr/git"</span><span class="token punctuation">,</span>
            <span class="token key atrule">"command"</span><span class="token punctuation">:</span> <span class="token punctuation">[</span>
                <span class="token string">"git"</span><span class="token punctuation">,</span>
                <span class="token string">"clone"</span><span class="token punctuation">,</span>
                <span class="token string">"https://github.com/mdn/beginner-html-site-scripted"</span><span class="token punctuation">,</span>
                <span class="token string">"/var/lib/data"</span>
            <span class="token punctuation">]</span><span class="token punctuation">,</span>
            <span class="token key atrule">"volumeMounts"</span><span class="token punctuation">:</span> <span class="token punctuation">[</span>
                <span class="token punctuation">{<!-- --></span>
                    <span class="token key atrule">"mountPath"</span><span class="token punctuation">:</span> <span class="token string">"/var/lib/data"</span><span class="token punctuation">,</span>
                    <span class="token key atrule">"name"</span><span class="token punctuation">:</span> <span class="token string">"git"</span>
                <span class="token punctuation">}</span>
            <span class="token punctuation">]</span>
        <span class="token punctuation">}</span>
    <span class="token punctuation">]</span>'
<span class="token key atrule">spec</span><span class="token punctuation">:</span>
  <span class="token key atrule">containers</span><span class="token punctuation">:</span>
  <span class="token punctuation">-</span> <span class="token key atrule">name</span><span class="token punctuation">:</span> run
    <span class="token key atrule">image</span><span class="token punctuation">:</span> docker.io/centos/httpd
    <span class="token key atrule">ports</span><span class="token punctuation">:</span>
      <span class="token punctuation">-</span> <span class="token key atrule">containerPort</span><span class="token punctuation">:</span> <span class="token number">80</span>
    <span class="token key atrule">volumeMounts</span><span class="token punctuation">:</span>
    <span class="token punctuation">-</span> <span class="token key atrule">mountPath</span><span class="token punctuation">:</span> /var/www/html
      <span class="token key atrule">name</span><span class="token punctuation">:</span> git
  <span class="token key atrule">volumes</span><span class="token punctuation">:</span>
  <span class="token punctuation">-</span> <span class="token key atrule">emptyDir</span><span class="token punctuation">:</span> <span class="token punctuation">{<!-- --></span><span class="token punctuation">}</span>
    <span class="token key atrule">name</span><span class="token punctuation">:</span> git
</code></pre> 
<p>这个例子就是用来再应用程序启动前首先从GitHub中拉取代码并存储到共享目录下。</p> 
<p>关于Init容器的更详细说明请参考 <a href="https://jimmysong.io/kubernetes-handbook/concepts/init-containers.html" rel="nofollow">init容器</a>。</p> 
<h3><a id="5__127"></a>5. 使容器内时间与宿主机同步</h3> 
<p>我们下载的很多容器内的时区都是格林尼治时间，与北京时间差8小时，这将导致容器内的日志和文件创建时间与实际时区不符，有两种方式解决这个问题：</p> 
<ul><li>修改镜像中的时区配置文件</li><li>将宿主机的时区配置文件<code>/etc/localtime</code>使用volume方式挂载到容器中</li></ul> 
<p>第二种方式比较简单，不需要重做镜像，只要在应用的yaml文件中增加如下配置：</p> 
<pre><code class="prism language-yaml"><span class="token key atrule">volumeMounts</span><span class="token punctuation">:</span>
  <span class="token punctuation">-</span> <span class="token key atrule">name</span><span class="token punctuation">:</span> host<span class="token punctuation">-</span>time
    <span class="token key atrule">mountPath</span><span class="token punctuation">:</span> /etc/localtime
    <span class="token key atrule">readOnly</span><span class="token punctuation">:</span> <span class="token boolean important">true</span>
  <span class="token key atrule">volumes</span><span class="token punctuation">:</span>
  <span class="token punctuation">-</span> <span class="token key atrule">name</span><span class="token punctuation">:</span> host<span class="token punctuation">-</span>time
    <span class="token key atrule">hostPath</span><span class="token punctuation">:</span>
      <span class="token key atrule">path</span><span class="token punctuation">:</span> /etc/localtime
</code></pre> 
<h3><a id="6_Podnamespace_147"></a>6. 在Pod中获取宿主机的主机名、namespace等</h3> 
<p>这条技巧补充了第一条获取 podIP 的内容，方法都是一样的，只不过列出了更多的引用字段。</p> 
<p>参考下面的 pod 定义，每个 pod 里都有一个 {.spec.nodeName} 字段，通过 <code>fieldRef</code> 和环境变量，就可以在Pod中获取宿主机的主机名（访问环境变量<code>MY_NODE_NAME</code>）。</p> 
<pre><code class="prism language-yaml"><span class="token key atrule">apiVersion</span><span class="token punctuation">:</span> v1
<span class="token key atrule">kind</span><span class="token punctuation">:</span> Pod
<span class="token key atrule">metadata</span><span class="token punctuation">:</span>
  <span class="token key atrule">name</span><span class="token punctuation">:</span> dapi<span class="token punctuation">-</span>test<span class="token punctuation">-</span>pod
<span class="token key atrule">spec</span><span class="token punctuation">:</span>
  <span class="token key atrule">containers</span><span class="token punctuation">:</span>
    <span class="token punctuation">-</span> <span class="token key atrule">name</span><span class="token punctuation">:</span> test<span class="token punctuation">-</span>container
      <span class="token key atrule">image</span><span class="token punctuation">:</span> busybox
      <span class="token key atrule">command</span><span class="token punctuation">:</span> <span class="token punctuation">[</span> <span class="token string">"/bin/sh"</span><span class="token punctuation">,</span> <span class="token string">"-c"</span><span class="token punctuation">,</span> <span class="token string">"env"</span> <span class="token punctuation">]</span>
      <span class="token key atrule">env</span><span class="token punctuation">:</span>
        <span class="token punctuation">-</span> <span class="token key atrule">name</span><span class="token punctuation">:</span> MY_NODE_NAME
          <span class="token key atrule">valueFrom</span><span class="token punctuation">:</span>
            <span class="token key atrule">fieldRef</span><span class="token punctuation">:</span>
              <span class="token key atrule">fieldPath</span><span class="token punctuation">:</span> spec.nodeName
        <span class="token punctuation">-</span> <span class="token key atrule">name</span><span class="token punctuation">:</span> MY_POD_NAME
          <span class="token key atrule">valueFrom</span><span class="token punctuation">:</span>
            <span class="token key atrule">fieldRef</span><span class="token punctuation">:</span>
              <span class="token key atrule">fieldPath</span><span class="token punctuation">:</span> metadata.name
        <span class="token punctuation">-</span> <span class="token key atrule">name</span><span class="token punctuation">:</span> MY_POD_NAMESPACE
          <span class="token key atrule">valueFrom</span><span class="token punctuation">:</span>
            <span class="token key atrule">fieldRef</span><span class="token punctuation">:</span>
              <span class="token key atrule">fieldPath</span><span class="token punctuation">:</span> metadata.namespace
        <span class="token punctuation">-</span> <span class="token key atrule">name</span><span class="token punctuation">:</span> MY_POD_IP
          <span class="token key atrule">valueFrom</span><span class="token punctuation">:</span>
            <span class="token key atrule">fieldRef</span><span class="token punctuation">:</span>
              <span class="token key atrule">fieldPath</span><span class="token punctuation">:</span> status.podIP
        <span class="token punctuation">-</span> <span class="token key atrule">name</span><span class="token punctuation">:</span> HOST_IP
          <span class="token key atrule">valueFrom</span><span class="token punctuation">:</span>
           <span class="token key atrule">fieldRef</span><span class="token punctuation">:</span>
             <span class="token key atrule">fieldPath</span><span class="token punctuation">:</span> status.hostIP
        <span class="token punctuation">-</span> <span class="token key atrule">name</span><span class="token punctuation">:</span> MY_POD_SERVICE_ACCOUNT
          <span class="token key atrule">valueFrom</span><span class="token punctuation">:</span>
            <span class="token key atrule">fieldRef</span><span class="token punctuation">:</span>
              <span class="token key atrule">fieldPath</span><span class="token punctuation">:</span> spec.serviceAccountName
  <span class="token key atrule">restartPolicy</span><span class="token punctuation">:</span> Never
</code></pre> 
<h3><a id="7_PodDNS_191"></a>7. 配置Pod使用外部DNS</h3> 
<p>修改kube-dns的使用的ConfigMap。</p> 
<pre><code class="prism language-yaml"><span class="token key atrule">apiVersion</span><span class="token punctuation">:</span> v1
<span class="token key atrule">kind</span><span class="token punctuation">:</span> ConfigMap
<span class="token key atrule">metadata</span><span class="token punctuation">:</span>
  <span class="token key atrule">name</span><span class="token punctuation">:</span> kube<span class="token punctuation">-</span>dns
  <span class="token key atrule">namespace</span><span class="token punctuation">:</span> kube<span class="token punctuation">-</span>system
<span class="token key atrule">data</span><span class="token punctuation">:</span>
  <span class="token key atrule">stubDomains</span><span class="token punctuation">:</span> <span class="token punctuation">|</span><span class="token scalar string">
    {"k8s.com": ["192.168.10.10"]}</span>
  <span class="token key atrule">upstreamNameservers</span><span class="token punctuation">:</span> <span class="token punctuation">|</span><span class="token scalar string">
    ["8.8.8.8", "8.8.4.4"]</span>
</code></pre> 
<p><code>upstreamNameservers</code> 即使用的外部DNS。</p> 
<h3><a id="8_CentOS_210"></a>8. 创建一个CentOS测试容器</h3> 
<p>有时我们可能需要在Kubernetes集群中创建一个容器来测试集群的状态或对其它容器进行操作，这时候我们需要一个操作节点，可以使用一个普通的CentOS容器来实现。yaml文件见<a href="https://github.com/rootsongjc/kubernetes-handbook/tree/master/manifests/test/centos.yaml">manifests/test/centos.yaml</a>。</p> 
<pre><code class="prism language-yaml"><span class="token key atrule">apiVersion</span><span class="token punctuation">:</span> extensions/v1beta1
<span class="token key atrule">kind</span><span class="token punctuation">:</span> Deployment
<span class="token key atrule">metadata</span><span class="token punctuation">:</span>
  <span class="token key atrule">name</span><span class="token punctuation">:</span> test
  <span class="token key atrule">labels</span><span class="token punctuation">:</span>
    <span class="token key atrule">app</span><span class="token punctuation">:</span> test
<span class="token key atrule">spec</span><span class="token punctuation">:</span>
  <span class="token key atrule">replicas</span><span class="token punctuation">:</span> <span class="token number">1</span>
  <span class="token key atrule">template</span><span class="token punctuation">:</span>
    <span class="token key atrule">metadata</span><span class="token punctuation">:</span>
      <span class="token key atrule">labels</span><span class="token punctuation">:</span>
        <span class="token key atrule">app</span><span class="token punctuation">:</span> test
    <span class="token key atrule">spec</span><span class="token punctuation">:</span>
      <span class="token key atrule">containers</span><span class="token punctuation">:</span>
      <span class="token punctuation">-</span> <span class="token key atrule">image</span><span class="token punctuation">:</span> harbor<span class="token punctuation">-</span>001.jimmysong.io/library/centos<span class="token punctuation">:</span>7.2.1511
        <span class="token key atrule">name</span><span class="token punctuation">:</span> test
        <span class="token key atrule">command</span><span class="token punctuation">:</span> <span class="token punctuation">[</span><span class="token string">"/bin/bash"</span><span class="token punctuation">,</span><span class="token string">"-c"</span><span class="token punctuation">,</span><span class="token string">"while true; do sleep 1000; done"</span><span class="token punctuation">]</span>
        <span class="token key atrule">imagePullPolicy</span><span class="token punctuation">:</span> IfNotPresent
</code></pre> 
<p>即使用一个<code>while</code>循环保证容器启动时拥有一个前台进程。</p> 
<p>也可以直接使用<code>kubectl run</code>的方式来创建：</p> 
<pre><code class="prism language-bash">kubectl run --image<span class="token operator">=</span>harbor-001.jimmysong.io/library/centos:7.2.1511 --command <span class="token string">'/bin/bash -c "while true;do sleep 1000;done"'</span> centos-test
</code></pre> 
<h3><a id="9_TerminatingPod_243"></a>9. 强制删除一直处于Terminating状态的Pod</h3> 
<p>有时候当我们直接删除Deployment/DaemonSets/StatefulSet等最高级别的Kubernetes资源对象时，会发现有些改对象管理的Pod一直处于Terminating而没有被删除的情况，这时候我们可以使用如下方式来强制删除它：</p> 
<p><strong>一、使用kubectl中的强制删除命令</strong></p> 
<pre><code class="prism language-bash">kubectl delete pod <span class="token variable">$POD_ID</span> --force --grace-period<span class="token operator">=</span><span class="token number">0</span>
</code></pre> 
<p>如果这种方式有效，那么恭喜你！如果仍然无效的话，请尝试下面第二种方法。</p> 
<p><strong>二、直接删除etcd中的数据</strong></p> 
<blockquote> 
 <p>这是一种最暴力的方式，我们不建议直接操作etcd中的数据，在操作前请确认知道你是在做什么。</p> 
</blockquote> 
<p>假如要删除<code>default</code> namespace下的pod名为<code>pod-to-be-deleted-0</code>，在etcd所在的节点上执行下面的命令，删除etcd中保存的该pod的元数据：</p> 
<pre><code class="prism language-bash"><span class="token assign-left variable">ETCDCTL_API</span><span class="token operator">=</span><span class="token number">3</span> etcdctl del /registry/pods/default/pod-to-be-deleted-0
</code></pre> 
<p>这时API server就不会再看到该pod的信息。</p>
                </div>
            </body>
            </html>
            